Oculomotor Remapping of Visual Information to Foveal Retinotopic Cortex.

Our eyes continually jump around the visual scene to bring the high-resolution, central part of our vision onto objects of interest. We are oblivious to these abrupt shifts, perceiving the visual world to appear reassuringly stable. A process called remapping has been proposed to mediate this perceptual stability for attended objects by shifting their retinotopic representation to compensate for the effects of the upcoming eye movement. In everyday vision, observers make goal-directed eye movements towards items of interest bringing them to the fovea and, for these items, the remapped activity should impinge on foveal regions of the retinotopic maps in visual cortex. Previous research has focused instead on remapping for targets that were not saccade goals, where activity is remapped to a new peripheral location rather than to the foveal representation. We used functional magnetic resonance imaging (fMRI) and a phase-encoding design to investigate remapping of spatial patterns of activity towards the fovea/parafovea for saccade targets that were removed prior to completion of the eye movement. We found strong evidence of foveal remapping in retinotopic visual areas, which failed to occur when observers merely attended to the same peripheral target without making eye movements towards it. Significantly, the spatial profile of the remapped response matched the orientation and size of the saccade target, and was appropriately scaled to reflect the retinal extent of the stimulus had it been foveated. We conclude that this remapping of spatially structured information to the fovea may serve as an important mechanism to support our world-centered sense of location across goal-directed eye movements under natural viewing conditions.

Radial bias is not necessary for orientation decoding.

Multivariate pattern analysis can be used to decode the orientation of a viewed grating from fMRI signals in early visual areas. Although some studies have reported identifying multiple sources of the orientation information that make decoding possible, a recent study argued that orientation decoding is only possible because of a single source: a coarse-scale retinotopically organized preference for radial orientations. Here we aim to resolve these discrepant findings. We show that there were subtle, but critical, experimental design choices that led to the erroneous conclusion that a radial bias is the only source of orientation information in fMRI signals. In particular, we show that the reliance on a fast temporal-encoding paradigm for spatial mapping can be problematic, as effects of space and time become conflated and lead to distorted estimates of a voxel's orientation or retinotopic preference. When we implement minor changes to the temporal paradigm or to the visual stimulus itself, by slowing the periodic rotation of the stimulus or by smoothing its contrast-energy profile, we find significant evidence of orientation information that does not originate from radial bias. In an additional block-paradigm experiment where space and time were not conflated, we apply a formal model comparison approach and find that many voxels exhibit more complex tuning properties than predicted by radial bias alone or in combination with other known coarse-scale biases. Our findings support the conclusion that radial bias is not necessary for orientation decoding. In addition, our study highlights potential limitations of using temporal phase-encoded fMRI designs for characterizing voxel tuning properties.

Auditory spatial attention representations in the human cerebral cortex.

Auditory spatial attention serves important functions in auditory source separation and selection. Although auditory spatial attention mechanisms have been generally investigated, the neural substrates encoding spatial information acted on by attention have not been identified in the human neocortex. We performed functional magnetic resonance imaging experiments to identify cortical regions that support auditory spatial attention and to test 2 hypotheses regarding the coding of auditory spatial attention: 1) auditory spatial attention might recruit the visuospatial maps of the intraparietal sulcus (IPS) to create multimodal spatial attention maps; 2) auditory spatial information might be encoded without explicit cortical maps. We mapped visuotopic IPS regions in individual subjects and measured auditory spatial attention effects within these regions of interest. Contrary to the multimodal map hypothesis, we observed that auditory spatial attentional modulations spared the visuotopic maps of IPS; the parietal regions activated by auditory attention lacked map structure. However, multivoxel pattern analysis revealed that the superior temporal gyrus and the supramarginal gyrus contained significant information about the direction of spatial attention. These findings support the hypothesis that auditory spatial information is coded without a cortical map representation. Our findings suggest that audiospatial and visuospatial attention utilize distinctly different spatial coding schemes.

How attention extracts objects from noise.

The visual system is remarkably proficient at extracting relevant object information from noisy, cluttered environments. Although attention is known to enhance sensory processing, the mechanisms by which attention extracts relevant information from noise are not well understood. According to the perceptual template model, attention may act to amplify responses to all visual input, or it may act as a noise filter, dampening responses to irrelevant visual noise. Amplification allows for improved performance in the absence of visual noise, whereas a noise-filtering mechanism can only improve performance if the target stimulus appears in noise. Here, we used fMRI to investigate how attention modulates cortical responses to objects at multiple levels of the visual pathway. Participants viewed images of faces, houses, chairs, and shoes, presented in various levels of visual noise. We used multivoxel pattern analysis to predict the viewed object category, for attended and unattended stimuli, from cortical activity patterns in individual visual areas. Early visual areas, V1 and V2, exhibited a benefit of attention only at high levels of visual noise, suggesting that attention operates via a noise-filtering mechanism at these early sites. By contrast, attention led to enhanced processing of noise-free images (i.e., amplification) only in higher visual areas, including area V4, fusiform face area, mid-Fusiform area, and the lateral occipital cortex. Together, these results suggest that attention improves people's ability to discriminate objects by de-noising visual input in early visual areas and amplifying this noise-reduced signal at higher stages of visual processing.

Perceptual learning selectively refines orientation representations in early visual cortex.

Although practice has long been known to improve perceptual performance, the neural basis of this improvement in humans remains unclear. Using fMRI in conjunction with a novel signal detection-based analysis, we show that extensive practice selectively enhances the neural representation of trained orientations in the human visual cortex. Twelve observers practiced discriminating small changes in the orientation of a laterally presented grating over 20 or more daily 1 h training sessions. Training on average led to a twofold improvement in discrimination sensitivity, specific to the trained orientation and the trained location, with minimal improvement found for untrained orthogonal orientations or for orientations presented in the untrained hemifield. We measured the strength of orientation-selective responses in individual voxels in early visual areas (V1-V4) using signal detection measures, both before and after training. Although the overall amplitude of the BOLD response was no greater after training, practice nonetheless specifically enhanced the neural representation of the trained orientation at the trained location. This training-specific enhancement of orientation-selective responses was observed in the primary visual cortex (V1) as well as higher extrastriate visual areas V2-V4, and moreover, reliably predicted individual differences in the behavioral effects of perceptual learning. These results demonstrate that extensive training can lead to targeted functional reorganization of the human visual cortex, refining the cortical representation of behaviorally relevant information.

Prevalence of selectivity for mirror-symmetric views of faces in the ventral and dorsal visual pathways.

Although the ability to recognize faces and objects from a variety of viewpoints is crucial to our everyday behavior, the underlying cortical mechanisms are not well understood. Recently, neurons in a face-selective region of the monkey temporal cortex were reported to be selective for mirror-symmetric viewing angles of faces as they were rotated in depth (Freiwald and Tsao, 2010). This property has been suggested to constitute a key computational step in achieving full view-invariance. Here, we measured functional magnetic resonance imaging activity in nine observers as they viewed upright or inverted faces presented at five different angles (-60, -30, 0, 30, and 60°). Using multivariate pattern analysis, we show that sensitivity to viewpoint mirror symmetry is widespread in the human visual system. The effect was observed in a large band of higher order visual areas, including the occipital face area, fusiform face area, lateral occipital cortex, mid fusiform, parahippocampal place area, and extending superiorly to encompass dorsal regions V3A/B and the posterior intraparietal sulcus. In contrast, early retinotopic regions V1-hV4 failed to exhibit sensitivity to viewpoint symmetry, as their responses could be largely explained by a computational model of low-level visual similarity. Our findings suggest that selectivity for mirror-symmetric viewing angles may constitute an intermediate-level processing step shared across multiple higher order areas of the ventral and dorsal streams, setting the stage for complete viewpoint-invariant representations at subsequent levels of visual processing.

Multishot versus single-shot pulse sequences in very high field fMRI: a comparison using retinotopic mapping.

High-resolution functional MRI is a leading application for very high field (7 Tesla) human MR imaging. Though higher field strengths promise improvements in signal-to-noise ratios (SNR) and BOLD contrast relative to fMRI at 3 Tesla, these benefits may be partially offset by accompanying increases in geometric distortion and other off-resonance effects. Such effects may be especially pronounced with the single-shot EPI pulse sequences typically used for fMRI at standard field strengths. As an alternative, one might consider multishot pulse sequences, which may lead to somewhat lower temporal SNR than standard EPI, but which are also often substantially less susceptible to off-resonance effects. Here we consider retinotopic mapping of human visual cortex as a practical test case by which to compare examples of these sequence types for high-resolution fMRI at 7 Tesla. We performed polar angle retinotopic mapping at each of 3 isotropic resolutions (2.0, 1.7, and 1.1 mm) using both accelerated single-shot 2D EPI and accelerated multishot 3D gradient-echo pulse sequences. We found that single-shot EPI indeed led to greater temporal SNR and contrast-to-noise ratios (CNR) than the multishot sequences. However, additional distortion correction in postprocessing was required in order to fully realize these advantages, particularly at higher resolutions. The retinotopic maps produced by both sequence types were qualitatively comparable, and showed equivalent test/retest reliability. Thus, when surface-based analyses are planned, or in other circumstances where geometric distortion is of particular concern, multishot pulse sequences could provide a viable alternative to single-shot EPI.

Multiscale pattern analysis of orientation-selective activity in the primary visual cortex.

Although orientation columns are less than a millimeter in width, recent neuroimaging studies indicate that viewed orientations can be decoded from cortical activity patterns sampled at relatively coarse resolutions of several millimeters. One proposal is that these differential signals arise from random spatial irregularities in the columnar map. However, direct support for this hypothesis has yet to be obtained. Here, we used high-field, high-resolution functional magnetic resonance imaging (fMRI) and multivariate pattern analysis to determine the spatial scales at which orientation-selective information can be found in the primary visual cortex (V1) of cats and humans. We applied a multiscale pattern analysis approach in which fine- and coarse-scale signals were first removed by ideal spatial lowpass and highpass filters, and the residual activity patterns then analyzed by linear classifiers. Cat visual cortex, imaged at 0.3125 mm resolution, showed a strong orientation signal at the scale of individual columns. Nonetheless, reliable orientation bias could still be found at spatial scales of several millimeters. In the human visual cortex, imaged at 1 mm resolution, a majority of orientation information was found on scales of millimeters, with small contributions from global spatial biases exceeding approximately 1 cm. Our high-resolution imaging results demonstrate a reliable millimeters-scale orientation signal, likely emerging from irregular spatial arrangements of orientation columns and their supporting vasculature. fMRI pattern analysis methods are thus likely to be sensitive to signals originating from other irregular columnar structures elsewhere in the brain.

Rapid and reversible recruitment of early visual cortex for touch.

BACKGROUND: The loss of vision has been associated with enhanced performance in non-visual tasks such as tactile discrimination and sound localization. Current evidence suggests that these functional gains are linked to the recruitment of the occipital visual cortex for non-visual processing, but the neurophysiological mechanisms underlying these crossmodal changes remain uncertain. One possible explanation is that visual deprivation is associated with an unmasking of non-visual input into visual cortex. METHODOLOGY/PRINCIPAL FINDINGS: We investigated the effect of sudden, complete and prolonged visual deprivation (five days) in normally sighted adult individuals while they were immersed in an intensive tactile training program. Following the five-day period, blindfolded subjects performed better on a Braille character discrimination task. In the blindfold group, serial fMRI scans revealed an increase in BOLD signal within the occipital cortex in response to tactile stimulation after five days of complete visual deprivation. This increase in signal was no longer present 24 hours after blindfold removal. Finally, reversible disruption of occipital cortex function on the fifth day (by repetitive transcranial magnetic stimulation; rTMS) impaired Braille character recognition ability in the blindfold group but not in non-blindfolded controls. This disruptive effect was no longer evident once the blindfold had been removed for 24 hours. CONCLUSIONS/SIGNIFICANCE: Overall, our findings suggest that sudden and complete visual deprivation in normally sighted individuals can lead to profound, but rapidly reversible, neuroplastic changes by which the occipital cortex becomes engaged in processing of non-visual information. The speed and dynamic nature of the observed changes suggests that normally inhibited or masked functions in the sighted are revealed by visual loss. The unmasking of pre-existing connections and shifts in connectivity represent rapid, early plastic changes, which presumably can lead, if sustained and reinforced, to slower developing, but more permanent structural changes, such as the establishment of new neural connections in the blind.

Working memory for social cues recruits orbitofrontal cortex and amygdala: a functional magnetic resonance imaging study of delayed matching to sample for emotional expressions.

During everyday interactions, we continuously monitor and maintain information about different individuals and their changing emotions in memory. Yet to date, working memory (WM) studies have primarily focused on mechanisms for maintaining face identity, but not emotional expression, and studies investigating the neural basis of emotion have focused on transient activity, not delay related activity. The goal of this functional magnetic resonance imaging study was to investigate WM for two critical social cues: identity and emotion. Subjects performed a delayed match-to-sample task that required them to match either the emotional expression or the identity of a face after a 10 s delay. Neuroanatomically, our predictions focused on the orbitofrontal cortex (OFC) and the amygdala, as these regions have previously been implicated in emotional processing and long-term memory, and studies have demonstrated sustained OFC and medial temporal lobe activity during visual WM. Consistent with previous studies, transient activity during the sample period representing emotion and identity was found in the superior temporal sulcus and inferior occipital cortex, respectively. Sustained delay-period activity was evident in OFC, amygdala, and hippocampus, for both emotion and identity trials. These results suggest that, although initial processing of emotion and identity is accomplished in anatomically segregated temporal and occipital regions, sustained delay related memory for these two critical features is held by the OFC, amygdala and hippocampus. These regions share rich connections, and have been shown previously to be necessary for binding features together in long-term memory. Our results suggest a role for these regions in active maintenance as well.

The distribution of category and location information across object-selective regions in human visual cortex.

Since Ungerleider and Mishkin [Underleider LG, Mishkin M (1982) Two cortical visual systems. Analysis of Visual Behavior, eds Ingle MA, Goodale MI, Masfield RJW (MIT Press, Cambridge, MA), pp 549-586] proposed separate visual pathways for processing object shape and location, steady progress has been made in characterizing the organization of the two kinds of information in extrastriate visual cortex in humans. However, to date, there has been no broad-based survey of category and location information across all major functionally defined object-selective regions. In this study, we used an fMRI region-of-interest (ROI) approach to identify eight regions characterized by their strong selectivity for particular object categories (faces, scenes, bodies, and objects). Participants viewed four types of stimuli (faces, scenes, bodies, and cars) appearing in each of three different spatial locations (above, below, or at fixation). Analyses based on the mean response and voxelwise patterns of response in each ROI reveal location information in almost all of the known object-selective regions. Furthermore, category and location information can be read out independently of one another such that most regions contain both position-invariant category information and category-invariant position information. Finally, we find substantially more location information in ROIs on the lateral than those on the ventral surface of the brain, even though these regions have equal amounts of category information. Although the presence of both location and category information in most object-selective regions argues against a strict physical separation of processing streams for object shape and location, the ability to extract position-invariant category information and category-invariant position information from the same neural population indicates that form and location information nonetheless remain functionally independent.

Visual topography of human intraparietal sulcus.

Human parietal cortex is implicated in a wide variety of sensory and cognitive functions, yet its precise organization remains unclear. Visual field maps provide a potential structural basis for descriptions of functional organization. Here, we detail the topography of a series of five maps of the contralateral visual hemifield within human posterior parietal cortex. These maps are located along the medial bank of the intraparietal sulcus (IPS) and are revealed by direct visual stimulation during functional magnetic resonance imaging, allowing these parietal regions to be routinely and reliably identified simultaneously with occipital visual areas. Two of these maps (IPS3 and IPS4) are novel, whereas two others (IPS1 and IPS2) have previously been revealed only by higher-order cognitive tasks. Area V7, a previously identified visual map, is observed to lie within posterior IPS and to share a foveal representation with IPS1. These parietal maps are reliably observed across scan sessions; however, their precise topography varies between individuals. The multimodal organization of posterior IPS mirrors this variability in visual topography, with complementary tactile activations found immediately adjacent to the visual maps both medially and laterally. These visual maps may provide a practical framework in which to characterize the functional organization of human IPS.

Combined activation and deactivation of visual cortex during tactile sensory processing.

The involvement of occipital cortex in sensory processing is not restricted solely to the visual modality. Tactile processing has been shown to modulate higher-order visual and multisensory integration areas in sighted as well as visually deprived subjects; however, the extent of involvement of early visual cortical areas remains unclear. To investigate this issue, we employed functional magnetic resonance imaging in normally sighted, briefly blindfolded subjects with well-defined visuotopic borders as they tactually explored and rated raised-dot patterns. Tactile task performance resulted in significant activation in primary visual cortex (V1) and deactivation of extrastriate cortical regions V2, V3, V3A, and hV4 with greater deactivation in dorsal subregions and higher visual areas. These results suggest that tactile processing affects occipital cortex via two distinct pathways: a suppressive top-down pathway descending through the visual cortical hierarchy and an excitatory pathway arising from outside the visual cortical hierarchy that drives area V1 directly.